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UPCONVERSION WITH NOISE CONSTRAINED DIAGONAL ENHANCEMENT 

Background of the Invention 

1. Technical Field 

This invention relates to the field of upconverting interlaced video to 
progressive (non-interlaced) video. More particularly, this invention relates to the 
field of deinterlacing algorithms. 

2. Description of Related Art 

When converting interlaced video to progressive scan format, a 
deinterlacing algorithm typically uses a temporal estimate (inter-field) in areas 
where no motion is detected, but uses a spatial estimate (intra-field) in motion 
areas. Spatial interpolation is used to generate an estimate for a missing line, 
based on the line above and below in the same field. 

A simple method of deinterlacing a field of luminance pixels to create lines 
of pixels that are spatially in between the existing lines of the field is to average the 
values of the pixel directly above and below to create each new pixel. However, 
this method produces jagged edges on diagonal details in the field, such as on 
steps that are not horizontal or the on edges of the stripes on a flag as it waves in 
the wind and the angles vary. 

A better method for enhanced diagonal detail, chooses between three pixel 
averages: the vertical average previously described and a left and right diagonal 
average. The left diagonal average is computed by averaging the value of the 
pixel to the left and above the position being interpolated with the pixel to the right 
and below. Likewise, the right diagonal is the average of the pixel to the right and 
above with the pixel to the left and below. 

The basic idea of selecting between two diagonal averages and a vertical 
average was explored by the Sarnoff Research Laboratories (Sarnoff) in a 
research project funded by Thomson (assignee herein) in the early 1990's. The 
simplest algorithm choosing between the three choices was denoted in their report 
as the DIAG1 algorithm. 

A simple algorithm to select one of these three averages was evaluated by 
Sarnoff and denoted DIAG1 . The DIAG1 algorithm for selecting between two 
diagonal averages and a vertical average computes and compares three 
differences using the same pixel values as the averages. For each average, the 
corresponding difference is computed and the absolute values of those differences 
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are compared to find a minimum. The pixel average that corresponds to the 
minimum difference is selected to be the interpolated value. 

The Samoff work mentioned is documented in two reports: Spatially- 
Adaptive De-Interlacing Techniques for Digital Feature TV, December 31, 1990 
and Digital Feature TV Project Final Report, March 31 , 1991 . 
Even with the apparent advantage of having multiple choices to consider for a 
spatial interpolation estimate, the existing decision process of DIAG1 appears 
somewhat random and noisy because the minimum value is not always the best 
choice. There is a long-felt need for an improved decision process that is relatively 
simple and inexpensive to implement, but nevertheless is less random and noisy. 

Summary of the Invention 

In accordance with the inventive arrangements, a noise constraint is added 
to the decision process of selecting between two diagonal averages and the 
vertical average, advantageously improving the picture quality in the areas where 
diagonal detail exists. This noise constraint causes the new algorithm, denoted 
DIAG1T, to prefer the vertical average in ambiguous cases. 

More particularly, the DIAG1T algorithm advantageously prefers the vertical 
average under two conditions, representing further decision steps in addition to the 
DIAG1 average selection algorithm. Firstly, if the minimum difference is not 
unique, the vertical average is selected. Secondly, if the selected average does 
not lie in the range of values between the pixel above and the pixel below the 
position being interpolated, then the vertical average is selected. 

The diagonal adjacent pixels line-up at angles in the displayed picture that 
correspond or depend upon on the sample rate. For a 720x480 picture with 4x3 
aspect ratio, for example, the diagonals correspond to angles of about 41 degrees 
above horizontal. The improved algorithm can improve diagonal features in the 
picture that are steeper than this angle. 

Brief Description of the Drawings 

Figure 1 is a block diagram of the DIAG1T diagonal enhancement 
upconversion system in accordance with the present invention. 

Figure 2 is a diagram useful for explaining the spatial orientation of original 
and interpolated pixels. 

Figure 3 is a block diagram illustrating the details of the Averaging and 
Difference (AVG-DIF) blocks in Figure 1. 
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Figure 4 is a block diagram illustrating the signal flow in the DIAG1T circuit 
shown in Figure 1 . 

Figure 5 is a block diagram of the DIAG1 decision logic shown in Figure 4. 
Figure 6 is a block diagram illustrating the DIAG1T constraint logic in Figure 

4. 

Detailed Description of the Preferred Embodiments 
The deinterlacing algorithm in accordance with the inventive arrangements 
is denoted herein as the DIAG1T. The DIAG1T deinterlacing algorithm can be 
used as a spatial only (intra-field) algorithm or as a spatial estimate in a motion 
adaptive deinterlacing algorithm. The improvement achieved by the DIAG1T 
algorithm, when used for the spatial estimate instead of a simple line average, is 
significant. 

In the following description, it is assumed that the video signal is in 
component form and that only the luminance component is processed with the 
D1AG1T algorithm. Thus, a "luminance pixel value" or "pixel value" as used herein 
refers to the luminance component. A simple line average is satisfactory for 
deinterlacing the two lower resolution chrominance components. 

Let the luminance pixel values on two consecutive lines of input video be 
labeled Xij and the interpolated progressive output line pixels be labeled Yij as 
follows, and as shown in Figure 2: 

Input line 1: X11 X12 X13 X14 X15 

Output line: Y11 Y12 Y13 Y14 Y15 

Input line 2: X21 X22 X23 X24 X25 
A simple line average spatial estimate would be: 

Y1j = (X1j + X2j)/2. 

The description of the DIAG1T algorithm given below will focus on 
computing the spatial estimate for the output position Y13. For other output pixels, 
the pixel indices in the description would be adjusted accordingly. An 
implementation of the algorithm can choose to modify the processing described 
herein at the beginnings and ends of lines when required pixels are not available. 

For Y13, the DIAG1T algorithm computes 3 pixel averages and 3 pixel 
differences as follows, and as shown in Figure 1 : 

y1L = (X12 + X24)/2; d1L = abs(X12 - X24); 
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y1V = (X13 + X23)/2; d1V = abs(X13 - X23); 

y1R = (X14 + X22)/2; d 1 R = abs(X1 4 - X22) ; 
These averages and differences correspond to a left diagonal, a vertical and a 
right diagonal estimate. An exemplary block diagram of a circuit 10 for providing 
all these averages and differences is shown in Figure 1 including a plurality of 
delay circuits 12, 14, 16, 18, and 20 in the form of flip flops and other suitable 
devices. For example, delay circuit 16 can be a line delay. The circuit further 
includes minimum circuit 22, maximum circuit 24, as well as AVG-DIFF blocks 26, 
28 and 30. The details of an exemplary AVG-DIF block such as the AVG-DIF 
block 26 in Figure 1 is shown in Figure 3. AVG-DIF block 26 preferably includes in 
an averaging portion of the device or block, a summer 34 for adding adjacent pixel 
values and a divide-by-two circuit 38. In a difference portion, block 26 further 
includes a subtractor 26 and an absolute value function 39. 

The basic DIAG1 algorithm chooses the estimate that corresponds to the 
minimum difference as follows, and as shown in Figure 5: 

y=y1V; d=d1V; 

if (d1L<d){y=y1L; d=d1L;} 

if(d1R<d) y=y1R; 

Y13=y; 

More specifically, the basic DIAG1 decision logic 42 can be represented 
using comparators 52 and 58 and multiplexers 54, 56 and 59 coupled as shown in 
Figure 5 to provide the functionality of the DIAG1 algorithm described above, 
although the present invention is not limited thereto. Note that the DIAG1 
algorithm or DIAG1 decision logic 42 forms a part of the overall DIAG1T decision 
logic 32 shown in Figures 1 and 4. 

The noise constraint or the D1AG1T constraint logic 44 shown in Figures 4 
and 6 that is used to clean up decision noise in DIAG1, and in so doing distinguish 
DIAG1T over DIAG1 , further insists that a value for the given output pixel (Y13) fall 
within a range of values of a top pixel and a bottom pixel defining the vertical 
average for the given output pixel. In terms of equations, this means that either: 

X13£Y13<X23 or X13£Y13>X23 
If this constraint is not satisfied, then output Y13 = y1 V. 
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An additional constraint distinguishing DIAG1T is that the left diagonal 
difference either equals or substantially equals the right diagonal difference. In 
other words, the differences for the left diagonally adjacent pixels and the right 
diagonally adjacent pixels need to be unique or d1 R must differ from d1L In terms 
of equations: 

if d1 L = d1 R , then output Y13 = y1 V. 
These further constraints are illustrated in Figures 4 and 6. In particular, the 
DIAG1T constraint logic 44 shown in Figure 6 can be embodied by comparators 
60, 62, and 64, OR gate 66 and multiplexer 68 arranged and coupled as shown to 
provide the functions described above. 

Thomson's Princeton Engine real time digital video simulator was 
programmed to demonstrate Sarnoffs DIAG1, DIAG3 and DIAG3W deinterlacing 
algorithms and Thomson's new DIAG1T deinterlacing algorithm. The Princeton 
Engine was programmed to perform each DIAG algorithm as a spatial only (intra- 
field) deinterlacer and also as the spatial estimate of a motion adaptive 
deinterlacing algorithm. 

The algorithms were simulated and demonstrated to various observers first 
with a fixed test pattern, the frame rate sweep, and then with frozen video scenes 
and finally with real time motion video. 

In a frame rate sweep test pattern, a wide variety of horizontal, vertical and 
diagonal frequencies exist. AH of the Sarnoff deinterlacing algorithms cause the 
deinterlaced test pattern to appear to have various regions defined by visible 
boundaries where some regions were estimated by the vertical average, and other 
regions by the left or right diagonal average. The main observable difference 
between the four algorithms DIAG1, DIAG3, DIAG3W and DIAG1T is the nature of 
the boundaries between the regions. When the frame rate sweep pattern is 
digitally generated, the boundaries are very clean and straight. However, when 
the test pattern is from an analog source with some noise, the boundaries are less 
well defined and jagged. With DIAG3W and DIAG1T the boundaries are 
noticeably cleaner. DIAG3 provides some improvement over DIAG1 . There is 
little difference between the appearance of DIAG3W and DIAG1T but the DIAG1T 
algorithm is much less complex and much easier to implement than DIAG3W. The 
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region boundaries are significantly cleaner with the DIAG1T algorithm than with the 
DIAG1 algorithm. 

In simulations with frozen or moving video, areas with diagonal detail are 
visibly improved. One such scene observed had an American flag waving in the 
wind. The stripes on the flag change from horizontal to various angles of diagonal 
as the flag moves in the wind and had visibly improved detail in such areas with 
diagonal detail. The improvement of jagged edges on some of the diagonal angles 
was significant. 

When used as the spatial estimate of a motion adaptive algorithm, the 
DIAG1T algorithm significantly improves moving areas of the picture without 
significantly adding complexity or cost. Stationary regions of the picture already 
have superior detail from adjacent fields of the video. 

In light of the foregoing description of the invention, it should be recognized that 
the present invention can be realized in hardware, software, or a combination of 
hardware and software. A method of interpolating a given output pixel value when 
upconverting interlaced video to progressive video according to the present 
invention can be realized in a centralized fashion in one processing system, or in a 
distributed fashion where different elements are spread across several 
interconnected systems. Any kind of computer system, or other apparatus adapted 
for carrying out the methods described herein, is suited. A typical combination of 
hardware and software could be a general purpose computer processor or digital 
signal processor with a computer program that, when being loaded and executed, 
controls the computer system such that it carries out the methods described 
herein. 

The present invention can also be embedded in a computer program 
product, which comprises all the features enabling the implementation of the 
methods described herein, and which, when loaded in a computer system, is able 
to carry out these methods. Computer program or application in the present 
context means any expression, in any language, code or notation, of a set of 
instructions intended to cause a system having an information processing 
capability to perform a particular function either directly or after either or both of the 
following a) conversion to another language, code or notation; b) reproduction in a 
different material form. 
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Additionally, the description above is intended by way of example only and 
is not intended to limit the present invention in any way, except as set forth in the 
following claims. 



